The malicious use or malfunction of advanced general-purpose AI (GPAI) poses risks that, according to leading experts, could lead to the 'marginalisation or extinction of humanity.' To address these risks, there are an increasing number of proposals for international agreements on AI safety. In this paper, we review recent (2023-) proposals, identifying areas of consensus and disagreement, and drawing on related literature to assess their feasibility. We focus our discussion on risk thresholds, regulations, types of international agreement and five related processes: building scientific consensus, standardisation, auditing, verification and incentivisation. Based on this review, we propose a treaty establishing a compute threshold above which development requires rigorous oversight. This treaty would mandate complementary audits of models, information security and governance practices, overseen by an international network of AI Safety Institutes (AISIs) with authority to pause development if risks are unacceptable. Our approach combines immediately implementable measures with a flexible structure that can adapt to ongoing research.

The United States, United Kingdom, European Union, and several other countries have signed an AI safety treaty laid out by the Council of Europe (COE), an international standards and human rights organization. This landmark treaty, known as the Framework Convention on artificial intelligence and human rights, democracy, and the rule of law, opened for signature in Vilnius, Lithuania. It is the first legally binding international agreement aimed at ensuring that AI systems align with democratic values. The treaty focuses on three main areas: protecting human rights (including privacy and preventing discrimination), safeguarding democracy, and upholding the rule of law. It also provides a legal framework covering the entire lifecycle of AI systems, promoting innovation, and managing potential risks.

Artificial Intelligence (AI) is playing an increasingly dominant role in the lives of people around the world. As yet, however, it is subject to remarkably little governance, let alone global governance. There is a strong case for the establishment of a UN regime on Artificial Intelligence. Such a regime should include a supervisory body that can provide a democratic input. Artificial Intelligence has immense value to offer to humanity, such as improved efficiency, new capabilities, and solutions for complex problems.